Skip to content

add DeepSpeedZeroConfig __repr__ method #596

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
jeffra merged 2 commits into from
Dec 11, 2020
Merged

add DeepSpeedZeroConfig __repr__ method #596

jeffra merged 2 commits into from
Dec 11, 2020

Conversation

@stas00
Copy link
Collaborator

@stas00 stas00 commented Dec 11, 2020 •
edited
Loading

DeepSpeedZeroConfig object isn't being shown during startup

before:

[2020-12-10 21:24:11,820] [INFO] [config.py:648:print]   zero_config .................. 
<deepspeed.runtime.zero.config.DeepSpeedZeroConfig object at 0x7f32d4c0dbb0>

after this PR:

[2020-12-10 21:59:46,612] [INFO] [config.py:648:print]   zero_config .................. {
    "allgather_bucket_size": 500000000,
    "allgather_partitions": true,
    "contiguous_gradients": false,
    "cpu_offload": false,
    "elastic_checkpoint": true,
    "load_from_fp32_weights": true,
    "overlap_comm": false,
    "reduce_bucket_size": 500000000,
    "reduce_scatter": true,
    "stage": 0
}

Also I'm not sure about other files, but this file is in DOS format (\r\n/^M line ending)... is this by design?

@jeffra jeffra merged commit 66268bd into deepspeedai:master Dec 11, 2020
@stas00 stas00 deleted the zero_config_repr branch December 11, 2020 20:46
@bobisapotato bobisapotato mentioned this pull request Jan 24, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@arashashari arashashari Awaiting requested review from arashashari

@awan-10 awan-10 Awaiting requested review from awan-10

@cli99 cli99 Awaiting requested review from cli99

@conglongli conglongli Awaiting requested review from conglongli

@eltonzheng eltonzheng Awaiting requested review from eltonzheng

@jeffra jeffra Awaiting requested review from jeffra

@minjiaz minjiaz Awaiting requested review from minjiaz

@niumanar niumanar Awaiting requested review from niumanar

@RezaYazdaniAminabadi RezaYazdaniAminabadi Awaiting requested review from RezaYazdaniAminabadi

@samyam samyam Awaiting requested review from samyam

@ShadenSmith ShadenSmith Awaiting requested review from ShadenSmith

@tjruwase tjruwase Awaiting requested review from tjruwase tjruwase is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@stas00 @jeffra